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By means of the concept of balanced estimation of diffusion entropy we evaluate reliable scale- 
invariance embedded in different sleep stages and stride records. Segments corresponding to Wake, 
light sleep, REM, and deep sleep stages are extracted from long-term EEG signals. For each stage 
the scaling value distributes in a considerable wide range, which tell us that the scaling behavior is 
subject- and sleep cycle- dependent. The average of the scaling exponent values for wake segments 
is almost the same with that for REM segments (~ 0.8). Wake and REM stages have significant 
high value of average scaling exponent, compared with that for light sleep stages (~ 0.7). For 
the stride series, the original diffusion entropy (DE) and balanced estimation of diffusion entropy 
(BEDE) give almost the same results for de-trended series. Evolutions of local scaling invariance 
show that the physiological states change abruptly, though in the experiments great efforts have 
been done to keep conditions unchanged. Global behaviors of a single physiological signal may lose 
rich information on physiological states. Methodologically, BEDE can evaluate with considerable 
precision scale- invariance in very short time series (~ 10 2 ), while the original DE method sometimes 
may underestimate scale- invariance exponents or even fail in detecting scale-invariant behavior. The 
BEDE method is sensitive to trends in time series. Existence of trend may leads to a unreasonable 
high value of scaling exponent, and consequent mistake conclusions. 

PACS numbers: 89.75.Fb, 05.45.-a, 05.40.-a 
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I. INTRODUCTION 

Scale-invariance embedded in physiological signals can 
shed light on mechanisms of dynamical processes occur- 
ring in human body, based upon which one can construct 
theoretical models of the processes and evaluate healthy 
states of disease suffers P, [|| . A typical example is the 
scaling behaviors in different sleep stages. A cycle of 
healthy sleep persists typically 1 to 2 hours, which con- 
stitutes a sequence of sleep stages including wake, light 
sleep, rapid eye movement (REM), and deep sleep. Little 
is known about the specific functions of these circadian 
rhythms, it is believed that the deep and REM sleep 
states are essential for physical recreation and memory 
reconsolidation, respectively. Extensive works show that 
heartbeat dynamics are characterized by long-range cor- 
relations and different long-range exponents are found 
for healthy and patients suffering from disease [3|, and 
for different sleep stages Q. In particular, at different 
sleep stages long-range correlation occurs solely during 
REM sleep, which is similar while less pronounced to 
that during wakefulness. Hence, scale-invariance embed- 
ded in heartbeat intervals can be used as an monitor 
of intrinsic neural- autonomic regulation of the circadian 
rhythms, which may find its potential use in disease di- 
agnosis and therapy. But evaluation of scale-invariance 
in physiological signals meets two challenges. 
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Methodologically, variance-based methods, such 
as Wavelet analysis [|| and De-trended fluctuation 
approach^, are widely used in literature to calculate 
scaling exponents. They can estimate correctly values 
of scaling exponents for fractional Brownian motions, 
but incorrectly for Levy walks, and even can not find 
out a scaling-invariant behavior existing in a Levy 
flight process due to divergency of the second moment 
[7|. A successful complementary method is diffusion 
entropy analysis. From a stationary time series, one 
can construct all the possible segments with a specified 
length. Regarding the length as time duration, each 
segment can be regarded as a trajectory of a particle 
starting from original point. The time series is then 
mapped to an ensemble with the trajectories being 
realizations of a stochastic motion. From distribution 
function of displacement one can calculate Shannon 
entropy, which is called diffusion entropy by Scafetta et 
al.[7j. Detailed works prove its powerful in evaluation 
of scaling exponents for both fractional Brownian and 
Levy motions Q. 

Practically, to obtain the probability distribution func- 
tion, we divide the distribution region of displacement 
into many bins and reckon the number of displacements 
occurring in each bin. The occurring probability at a 
bin is generally approximated with relative frequency, 
namely, ratio between the occurring number and the total 
number of realizations, which is perfect for an ensemble 
with infinite number of realizations. However, a physio- 
logical signal is generally very short. Sometimes we can 
obtain a long time series, but there occur some phase 
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transitions in the measuring duration, which separate the 
series into short segments with different scaling behav- 
iors, respectively. For example, a typical cycle of healthy 
sleep contains several thousands of heartbeat intervals, 
in which a sequence of transitions occur between differ- 
ent sleep stages. Figure 2(a) presents a long-term heart- 
beat interval series for a healthy subject. The lengths for 
wake, REM, light sleep, and deep sleep stages distribute 
in wide regions from 10 1 to 10 3 without characteristic 
lengths, respectively (see materials). 

Short length of time series may induce large statisti- 
cal fluctuations and/or bias to physical quantities such 
as probability, moment, and entropy @. In a recent pa- 
per, Bonachela et al. [T(j] review the efforts for improved 
estimators of entropy for small data sets and propose ac- 
cordingly a balanced estimator that performs well when 
the data sets are small. In one of our recent papers, we 
propose a new concept called balanced estimator of diffu- 
sion entropy (BEDE) [ll|, in which the original form of 
entropy in diffusion entropy analysis is replaced with the 
balanced estimator of entropy. Calculations show that it 
gives reliable scaling exponents for short time series with 
length ~ 10 2 . 

In the present paper, the BEDE method is used to 
evaluate scaling behaviors in heartbeat series for differ- 
ent sleep stages and stride time series for normal, fast, 
and slow walkers. Results show that for finite records 
of physiological signals the current methods in literature 
may lead to unacceptable errors for scaling exponents 
and wrong conclusions, while the BEDE approach can 
provide us a reliable estimation of scaling exponents. 



II. METHOD AND MATERIALS 

A. Diffusion Entropy 

Let us review briefly the concept of diffusion entropy 
proposed to detect scale-invariance in stationary se- 
ries. For a stationary time series, £,1,^2, • ■ • ,6v, all the 
possible segments with specified length s read 



X i (s) = {6,6+1 



,6i+s-l},« 



1,2,. 



,N 



1. 



(1) 

We regard the length s as time and consequently Xi(s) 
the zth realization of a stochastic process. The total 
N — s + 1 realizations form an ensemble of the process. 
Displacement of the zth realization read 



Xi(s) 



!+S— 1 

E 



(2) 



Dividing the interval the displacements occur into M(s) 
bins, one can reckon the number of displacements occur- 
ring in each bin, denoted with n(k, s), k = 1, 2, • • • , M(s). 
The probability distribution function can be approxi- 
mated with the relative frequency 

n(k, s) 



p{k,s) ~j3(fc,s) 



N-s+1 



,fc = l,2,.-- ,M(s). (3) 



The consequent naive approximation of Shannon entropy 
reads 



M(s) 

p(k, s)ln\p(k,s)]. (4) 



fc=i 



Provided the stochastic process behaves scale- 
invariant, we have 



p (k,s)~j,F^ 



(s) + (fc-0.5)e(s) 



k= 1,2,-,M( S ), 



(5) 



where e(s) is the size of bin, which is simply selected to 
be a certain fraction of standard deviation of the initial 
series. Plugging Eq.(5) into Eq.(4) leads to 

/+00 
dyF(y)ln[F(y)] + Sln(s) = A + Sln(s), 
-00 

(6) 

As a powerful method, diffusion entropy (DE) has been 
used to evaluate scaling invariance embedded in time se- 
ries in diverse fields, such as solar activities spec- 
tra of complex networks fisf . physiological signals 3], 
DNA sequences [l5j. geographical phenomena [l6j], and 
finance 17j . 



B. Balanced Estimation of Diffusion Entropy 

Unfortunately, extension of Eq.(6) to the naive approx- 
imation of diffusion entropy is a nontrivial step, i.e., gen- 
erally S%% ve (s) ^A + Sln(s) Q. Defining relative error, 
fx(k,s) = ^ fc 'p(fcs) fc '^ ; after a straightforward computa- 
tion we have, 

S DE ( S ) = S n D T e + 2( ^ ( ! ) 7 + 1 1) + 0{M(s)}. (7) 

The leading order of error, ^fS^Tl , vanishes as N — s — > 
00, while it may become unacceptable large when N — s 
is finite. Especially, for short time series the linear rela- 
tion in Eq.(6) will be distorted completely and one can 
not find scaling invariant behavior. In the naive approx- 
imation of diffusion entropy there exist simultaneously 
statistical error (variance) and systematical error (bias). 
That is, p(k, s)ln[p(k, s)} is not an acceptable estimation 
of its corresponding term p(k, s)ln\p(k, s)]. Hence, our 
task is to find a new estimation of p(k, s)ln[p(k, s)], de- 
noted with Sd e [n{k, s)] , which make combination of vari- 
ance and bias minimum. Here, we ignore correlations 
between n(k, s), k = 1, 2, • • ■ , M(s). 

We employ the solution proposed by Bonachela et al. 
10]. Mathematically, this problem can be formulated as, 



dA 2 (k,s) _ 



0. 



dSDE[n(k,s)] 

A 2 (fc, s) = £ [Al as (k, s) + Al at (k, s)] w\p(k, s)]dp(k, s), 

(8) 
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where 

A L S (M = (p{k,s)ln\p{k,s)] - (s DE [n(k,8)] 
A 2 stat (k,s) = {(s DE [n(k,s)} - (s DE {n(k,s)} 

' (9) 

are bias and variance, respectively. And w[p{k, s)] is 
weight function depending on specific problem. Gener- 
ally, we set w[p(k, s)} = 1 due to lack of extra knowledge. 
The average < . > is conducted by using binomial distri- 
bution function, 

Pn(J,s) \p(j, s )} = n{j.s)\[N-s+l-nU,s)]< X Qg\ 

A simple computation leads to 11], 

( ■ \ i 1 N-s+3 

fe=n0',s)+2 



Consequently, a proper estimation of Sde(s) reads, 

, M(s) N-s+3 

i=l fc=n(j,s)+2 

called Balanced Estimator of Diffusion Entropy (BEDE) . 

C. Materials 

We consider long-term EEG signals for a total of 16 
male subjects aged from 32 to 56 (mean age 43), with 
weights from 89 to 152kg (mean weight 119kg) [18]. Each 
EEG record persists averagely 7.5 hours annotated with 
sleep staging and apnea information. Each annotation 
applies to thirty seconds following it. Sleep stages are 
divided into four stages, namely, deep sleep, light sleep, 
REM sleep, and wake phase, which are determined by 
using visual evaluation of electrophysiological recordings 
of brain activity. 

We consider also stride series for a total of 10 young 
healthy volunteers, denoted with siOl, si02, • ■ • , silO [lj|. 
Healthy here refers to that the participants have not his- 
tory of any neuromusucular, respiratory, or cardiovas- 
cular disorders and are taking no medication. Age dis- 
tributes in 18 to 29 years. Average age is 21.7 years. 
Height and weight center at 177cm and 71.8kg, with 
standard deviations 8cm and 10.7kg, respectively. All 
the subjects walk continuously on level ground around 
an obstacle free, long (either 225m or 400m), approx- 
imately oval path. The stride interval is measured by 
using ultra-thin, force sensitive switches taped inside 
one shoe. Each subject walks with four trials, includ- 
ing slow, normal, fast, and metro-regulated. For the 
slow, normal, and fast trials the mean stride intervals are 
1.3±0.2m, l.lztO.lm and 1.0±0.1m, and the mean walk- 
ing rates are 1.0±0.2m/s, 1.4±0.1m/s and 1.7±0.1m/s, 
respectively. 



Scale-invariance embedded in heartbeat/stride inter- 
val series, denoted with {yf, , ■ • ■ ,y%}, are evaluated. 
The key step in conducting BEDE is to guarantee the 
considered series being stationary [201 ] . The centered 
moving average method [21( is employed to obtain trend 
of a time series, namely, from the original series we cal- 
culate its trend y T , where the elements read, 



Vi — s '^=-[(«+l)/2]+li/»+i' 

* = [(* + l)/2], [(a + l)/2] + 1, • 



,N-[s/2}. 



(13) 



Herein, the size of moving window is identical with s in 
Sde(s). The consequent de-trended series y D read, 



y 



,D 



y, 



T ■ 

-Vi ,« 



[(s + l)/2],[(s+l)/2] + l, 



,N-[s/2]. 
(14) 

Fractional Brownian motions are generated to investi- 
gate the performance of centered moving average. Figure 
l(a)-(c) present results for three series with Hurst expo- 
nents H = 0.9, 0.7 and 0.3, as typical examples. DE and 
BEDE methods are used to estimate 8 values for the orig- 
inal signals and the corresponding de-trended series. The 
curves show that from the de-trended series BEDE can 
obtain almost the same values of 8 compared with that 
from the original series, namely, in the BEDE method 
the centered moving average does not introduce artifi- 
cial characteristics. From the original series, BEDE can 
obtain correctly values of 8, while DE method can not 
find scaling invariance in signals with larger values of H 
(the curves bend down, especially for signals with large 
H). Fig. 1(d) show the variance and bias (mean) of esti- 
mated exponents for de-trended series by using DE and 
BEDE, respectively. The average is conducted over 1000 
series with length 300 for each H . One can find that the 
BEDE method can estimate 8 for signals with < 8 < 1 
without bias and with higher precision, while DE method 
underestimates 8 up to 10% and has comparatively lower 
precision. 



III. RESULTS 
A. Scaling Behaviors for Sleep Stages 

Figure 2(a) shows the heartbeat interval series for sub- 
ject numbered 59 as a typical record (part). One can find 
that transition between different sleep stages occurs fre- 
quently, as annotated in Fig. 2(b). There exists a compli- 
cated and significant trend (see Fig. 2(c), in which trend 
for s — 21 is shown as an example). The corresponding 
de-trended series is depicted in Fig. 2(d). From all the 
records one can find that most of lengths for wake, light 
sleep, REM, and deep sleep stages distribute in 10 1 ~ 10 3 
(without characteristic lengths), as presented in Fig. 2(e). 

As a typical example, we show in Fig.3(a)-(d) results 
for the segments 7290s -8520s (wake), 8520s -9270s (light 
sleep), 9270s - lOHOs(REM), and 6450s - 7290s(deep 
sleep), which forms a sleep cycle of the subject num- 
bered 59, respectively. One can find that the curves for 
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FIG. 1: (Color online) Performance of centered mov- 
ing average. The series are fractional Brownian motions 
with Length 300. (a)-(c) Results for three series with 
H = 0.9,0.7, and 0.3, respectively. BEDE can accu- 
rately estimate S from the original and de-trended series. 
DE curve for the original series bend down. DE can 
detect scale-invariance in the de-trended series. Confi- 
dent intervals for estimated values are all less than 0.02. 
(d) Bias and deviation of the DE and BEDE estimations 
for de-trended series. BEDE can estimate scaling expo- 
nent without bias and with higher precision, while DE 
underestimates scaling exponent up to 10% with lower 
precision. 
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FIG. 2: (Color online) EEG records, (a) Heartbeat in- 
terval series for subject numbered 59, part is shown as 
an example, (b) Sleep stages annotated by visual evalua- 
tion of electrophysiological recordings of brain activity, (c) 
Trend extracted from the original series in (a). Trend for 
s = 21 is shown as an example, (d) De-trended series 
for s — 21 as an example, (e) Lengths for wake, light 
sleep, REM, and deep sleep stages distribute in 10 1 ~ 10 3 
(without characteristic lengths). 
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FIG. 3: (Color online) Scaling behaviors in a sleep cycle 
of the subject numbered 59. (a)-(d) Scaling behaviors of 
the segments 7290s - 8520s(wake), 8520s - 9270s(light 
sleep), 9270s - lOHOs(REM), and 6450s - 7290s(deep 
sleep), respectively. Confident intervals for estimated val- 
ues are all less than 0.02. 
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FIG. 4: (Color online) Distribution of scaling exponents 
for different sleep stages. From all the subjects the seg- 
ments corresponding to wake, light sleep, REM, and deep 
sleep are extracted, respectively. Segments whose lengths 
are less than 500 are discarded, (a)-(d) Scaling exponent 
distributions for wake, light sleep, REM, and deep sleep, 
respectively. The average values for REM and wake are 
significantly large (~ 0.8) compared with that for light 
sleep (~ 0.7). 
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DE results bend down with the increase of scale, while 
this trend are corrected by BEDE to straight lines in 
a considerable range of scale. What is more, though 
BEDE can detect successfully scaling behaviors in the 
original series, the estimated values of scaling exponents 
are significantly large compared with that for the corre- 
sponding de-trended series. From the original series the 
estimations for wake, light sleep, REM, and deep sleep 
are 1.00, 0.91, 1.07, 0.92, while that from the correspond- 
ing de-trended series are 0.82, 0.70, 0.92, 0.93. For the de- 
trended series, DE gives underestimates of the exponents 
up to 10%. Hence, to obtain reliable scaling behaviors for 
the different sleep stages we must conduct de-trend pro- 
cedure and use BEDE instead of DE. Results for shuffled 
de-trended series tell us that the scaling behaviors come 
from non-trivial patterns in series rather than distribu- 
tion of elements in series. 

From all the subjects we extract the segments cor- 
responding to wake, light sleep, REM, and deep sleep, 
whose lengths are larger than 500 (about a duration of 
450s), the number of which are 41,105,18 and 17, re- 
spectively. The distribution behaviors of the values for 
scaling exponent are presented in Fig.4(a)-(d). One can 
find that the average value and standard deviation of 
scaling exponent for REM are almost the same with that 
for wake, but the distribution for the former one is much 
sharper than that for the later one. The average value 
and standard deviation for light sleep are almost identi- 
cal with that for deep sleep, while the detailed shapes for 
the distributions are different completely. Obviously, to 
confirm if the differences in distributions originate from 
intrinsic behaviors or just from statistical errors, requires 
collection of a large amount of cases. What is more, the 
average values for REM and wake are significantly large 
(<~ 0.8) compared with that for light sleep (~ 0.7). 



B. Scaling Behaviors for Stride Series 

From stride records, one can calculate the correspond- 
ing stride interval series, trends, and de-trended se- 
ries. As an example, see in Fig. 5(a) results from nor- 
mal walk record of the volunteer numbered szOl. For 
slow, normal, and fast walks the lengths of the series 
are 3304,3371,3395, respectively, which are significantly 
larger than that of the sleep stages. For the original series 
with the increase of scale the DE curves bend down, i.e., 
scaling behavior can not be detected successfully, while 
the BEDE curves are almost perfect straight lines in con- 
siderable wide scales, as shown in Fig.5(b)-(d). However, 
the estimated values of scaling exponent from the origi- 
nal series maybe unacceptable large compared with that 
from the corresponding de-trended series (e.g., the dif- 
ference for slow record is 0.10). Consequently, de-trend 
procedure is the key step to obtain reliable estimations 
of scaling exponent. What is more, the estimated values 
by using DE are very close to (generally smaller than) 
that by using BEDE, and this conclusion stands for all 
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FIG. 5: (Color online) Scale-invariance of stride series, 
(a) Stride interval series from normal walk record of the 
volunteer numbered siOl. And the corresponding trend 
and de-trended series. Trend and de-trend series for s=21 
are shown as an example, (b)-(d) Scaling behaviors in 
slow, normal, and fast walk series. For the de-trended 
series, the estimated values by using DE are very close to 
(generally smaller than) that by using BEDE. Confident 
intervals for estimated values are all less than 0.02. 
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FIG. 6: (Color online) The BEDE results of de-trended 
series for all the ten volunteers, (a)-(c) Scaling behaviors 
embedded in slow, normal, and fast series (de-trended). 
Confident intervals for estimated values are all less than 

0.02. 
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FIG. 7: (Color online) Evolution of scaling behaviors 
in stride records, (a)-(c) Scaling behaviors embedded 
in slow, normal, and fast walk records of the volunteer 
numbered siOl. The scaling exponents oscillate abruptly 
in wide ranges. At = 700 
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FIG. 8: (Color online) Distribution details of the evo- 
lutionary scaling exponents for slow, normal , and fast 
walking series of all the ten volunteers. Details of the 
distributions are different completely. At = 700. 



the ten volunteers (not presented). 

The BEDE results of dc-trendcd series, y D , for all the 
ten volunteers are shown in Fig. 6. The BEDE curves 
are almost perfect straight lines in considerable width of 
scale, and the estimations of scaling exponents, namely, 
slopes of the curves, are also presented. 

An interesting question is if the physiological states 
of the volunteers keep unchanged in the walking ex- 
periments. Sliding a window along a de-trended se- 
ries, at the r'th step the window covers segment 
y^, Vt+Ii " ■> 2/t+At-u wnere At is size of the window. 
Representing the local scaling behavior at step t by the 
scaling exponent for the segment covered by the win- 
dow, denoted with S EEDE (t) , the successive values of 
6$ ede (t),t = 1,2, ••• ,N - At + 1 present the evo- 
lution of scaling behavior of the considered series. As 
a typical result, Fig.7(a)-(c) provide evolutions of scal- 
ing behaviors embedded in slow, normal, and fast walk 
records of the volunteer numbered siOl. Fig. 8 provides 
the distribution details, mean and standard values of the 
evolutionary scaling exponents for slow, normal , and 
fast walking series of all the volunteers. Unexpectedly, 
the scaling exponents oscillate abruptly in wide ranges 
and the details of local scaling exponent distributions 
are different completely. At is chosen to be 700 in the 
calculations. 



IV. CONCLUSIONS 

Scale-invariance in physiological signal records attracts 
special attentions for its importance in understanding 
and consequent modeling mechanisms of physiological 
phenomena and its potential usage in diagnoses and ther- 
apy. But evaluation of scale-invariance in physiological 
signal is a non-trivial task. Theoretically, variance-based 
methods may lead to a failure evaluation of scaling behav- 
ior. Practically, a physiological signal generally is itself 
very short (~ 10 2 ), or separated by frequent occurrences 
of phase transitions into short segments. In literature, 
persistent efforts have been done to find reliable meth- 
ods to evaluate scale-invariance in short time series (22[. 

In the present paper, we extract wake, light sleep, 
REM, and deep sleep segments (length > 500 for reliable 
estimation of scaling exponent) from long-term EEG sig- 
nals. By means of the concept of balanced estimation of 
diffusion entropy (BEDE) , we estimate the scaling expo- 
nents of de-trended series constructed from the segments. 
It is found that for each stage the scaling value distributes 
in a considerable wide range, i.e., the scaling behavior is 
subject- and sleep cycle- dependent. Statistically, the av- 
erage of the scaling exponent values for wake segments 
is almost the same with that for REM segments (~ 0.8), 
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while the average of the scaling exponent values for light 
sleep segments is about ~ 0.7. 

For the stride scries, because the series are long enough 
(3000 - 4000), the original diffusion entropy (DE) and 
balanced estimation of diffusion entropy (BEDE) give al- 
most the same results for de-trended series. But from the 
evolutions of local scaling invariance one can find that 
the physiological states change abruptly, though in the 
experiments great efforts have been done to keep con- 
ditions unchanged. Hence, global behaviors of a single 
physiological signal may lose rich information on physio- 
logical states. 

Comparison of the results of DE and BEDE, one can 
find that BEDE can evaluate with considerable precision 
scale-invariance in very short time series (~ 10 2 ). The 
original DE method sometimes underestimates values of 
scaling exponent, or even can not detect the scaling- 
behavior due to the bias of bending down. The two meth- 
ods (BEDE and DE) are all sensitive to trends in time 
series. In BEDE concept, existence of trend may leads to 
a unreasonable high value of scaling exponent, which may 



lead to mistake conclusions. Hence, balanced estimation 
of diffusion entropy (BEDE) is a preferential candidate to 
evaluate correctly and precisely scale-invariance in short 
time series, provided that the time series is de-trended 
properly. 
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